Method and system for optical imaging using patterned illumination

ABSTRACT

Systems and methods for optical imaging are disclosed. The systems and methods include a display for imaging an input object. The display includes a sensing surface; a plurality of display pixels; a plurality of detector pixels; and a processing system. The processing system is configured to determine a location of the input object relative to the sensing surface; illuminate one or more display pixels of the plurality of display pixels according to a pattern depending on the location of the input object; and acquire image data of the input object from one or more detector pixels of the plurality of detector pixels, wherein the image data corresponds to light from the one or more display pixels that is reflected at the sensing surface.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/966,806, filed Dec. 11, 2015, the contents of which are expresslyincorporated by reference.

FIELD

This disclosure generally relates to optical imaging, and moreparticularly optical imaging using patterned illumination in a display.

BACKGROUND

Object imaging is useful in a variety of applications. By way ofexample, biometric recognition systems image biometric objects forauthenticating and/or verifying users of devices incorporating therecognition systems. Biometric imaging provides a reliable,non-intrusive way to verify individual identity for recognitionpurposes. Various types of sensors may be used for biometric imaging.

Fingerprints, like various other biometric characteristics, are based ondistinctive personal characteristics and thus provide a reliablemechanism to recognize an individual. Thus, fingerprint sensors havemany potential applications. For example, fingerprint sensors may beused to provide access control in stationary applications, such assecurity checkpoints. Fingerprint sensors may also be used to provideaccess control in mobile devices, such as cell phones, wearable smartdevices (e.g., smart watches and activity trackers), tablet computers,personal data assistants (PDAs), navigation devices, and portable gamingdevices.

Most commercially available fingerprint sensors are based on optical orcapacitive sensing technologies. Unfortunately, conventional opticalfingerprint sensors are too bulky to be packaged in mobile devices andother common consumer electronic devices, confining their use to dooraccess control terminals and similar applications where sensor size isnot a restriction. As a result, fingerprint sensors in most mobiledevices are capacitive sensors having a sensing array configured tosense ridge and valley features of a fingerprint. Typically, thesefingerprint sensors either detect absolute capacitance (sometimes knownas “self-capacitance”) or trans-capacitance (sometimes known as “mutualcapacitance”). In either case, capacitance at each pixel in the arrayvaries depending on whether a ridge or valley is present, and thesevariations are electrically detected to form an image of thefingerprint.

While capacitive fingerprint sensors provide certain advantages, mostcommercially available capacitive fingerprint sensors have difficultysensing fine ridge and valley features through large distances,requiring the fingerprint to contact a sensing surface that is close tothe sensing array. It remains a significant challenge for a capacitivesensor to detect fingerprints through thick layers, such as the thickcover glass (sometimes referred to herein as a “cover lens”) thatprotects the display of many smart phones and other mobile devices. Toaddress this issue, a cutout is often formed in the cover glass in anarea beside the display, and a discrete capacitive fingerprint sensor(often integrated with a mechanical button) is placed in the cutout areaso that it can detect fingerprints without having to sense through thecover glass. The need for a cutout makes it difficult to form a flushsurface on the face of device, detracting from the user experience, andcomplicating the manufacture. The existence of mechanical buttons alsotakes up valuable device real estate.

SUMMARY

One embodiment of the disclosure provides an electronic device forimaging an input object. The electronic device includes a displaycomprising a sensing surface and an array of display pixels. Theelectronic device also includes a processing system communicativelycoupled to the display, the processing system configured to: selectivelyilluminate one or more of the display pixels according to a pattern;acquire image data, from one or more detector pixels of the display, ofthe input object in contact with the sensing surface, wherein the imagedata corresponds to light from the illuminated display pixels that isreflected at the sensing surface of the display; and process an image ofthe input object from the image data based upon the pattern.

Another embodiment of the disclosure provides an electronic device forimaging an input object. The electronic device includes a processingsystem configured to be communicatively coupled to a display. Theprocessing system is configured to selectively illuminate one or moredisplay pixels of the display according to a pattern; acquire imagedata, from one or more detector pixels of the display, of an inputobject in contact with a sensing surface of the display, wherein theimage data corresponds to light from the illuminated display pixels thatis reflected at the sensing surface of the display; and process an imageof the input object from the image data based upon the pattern.

Another embodiment of the disclosure provides a method for imaging aninput object. The method includes selectively illuminating one or moredisplay pixels of a display according to a pattern; acquiring imagedata, at one or more detector pixels of the display, of the input objectin contact with a sensing surface of the display, wherein the image datacorresponds to light from the illuminated display pixels that isreflected at the sensing surface of the display; and processing an imageof the object from the image data based on the pattern.

Another embodiment of the disclosure provides an electronic device forimaging a fingerprint. The electronic device includes a displaycomprising a sensing surface, an array of display pixels and an array ofdetector pixels. The electronic device also includes a processing systemcommunicatively coupled to the display, wherein the processing system isconfigured to: detect the fingerprint in contact with the sensingsurface; selectively illuminate one or more of the display pixelsaccording to a pattern; acquire image data of the fingerprint at one ormore of the detector pixels according to the pattern; and compare theacquired image data of the fingerprint to stored image data.

Another embodiment of the disclosure provides an electronic device forimaging a fingerprint. The electronic device includes a displaycomprising a sensing surface, an array of display pixels and an array ofdetector pixels and a processing system communicatively coupled to thedisplay. The processing system configured to illuminate a first set ofone or more of the display pixels and illuminate a second set of one ormore of the display pixels; acquire partial image data from a first setof the detector pixels during illumination of the first set of one ormore display pixels and acquire partial image data from a second set ofthe detector pixels during illumination of the second set of one or moredisplay pixels, wherein the first set of detector pixels is selectedfrom a first region surrounding the first set of one or more displaypixels, wherein the second set of detector pixels is selected from asecond region surrounding the second set of one or more display pixels;combine the partial image data from the first set of detector pixels andthe partial image data from the second set of detector pixels intocomposite fingerprint image data; and compare the composite fingerprintimage to an enrolled fingerprint template to determine a biometricmatch.

Another embodiment of the disclosure provides a method for imaging afingerprint. The method includes illuminating a first set of one or moredisplay pixels of a display and illuminating a second set of one or moredisplay pixels of the display; acquiring partial image data from a firstset of detector pixels of the display during illumination of the firstset of one or more display pixels and acquiring partial image data froma second set of detector pixels of the display during illumination ofthe second set of one or more display pixels, wherein the first set ofdetector pixels is selected from a first region surrounding the firstset of one or more display pixels, wherein the second set of detectorpixels is selected from a second region surrounding the second set ofone or more display pixels; combining the partial image data from thefirst set of detector pixels and the partial image data from the secondset of detector pixels into composite fingerprint image data; andcomparing the composite fingerprint image to an enrolled fingerprinttemplate to determine a biometric match.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example of a system that includes adisplay device and a processing system, according to an embodiment ofthe disclosure.

FIG. 2 illustrates a display according to an embodiment of thedisclosure.

FIG. 3 illustrates an array of sensor pixels and an array of displaypixels operating in a pattern according to an embodiment of thedisclosure.

FIGS. 4A-4B illustrate an array of sensor pixels and an array of displaypixels operating in a pattern according to an embodiment of thedisclosure.

FIG. 5 illustrates a portion of a fingerprint superimposed onto adetection region according to an embodiment of the disclosure.

FIG. 6 illustrates a display providing visual feedback during opticalimaging of an input object according to an embodiment of the disclosure.

FIG. 7 illustrates a method of optical imaging using patternedillumination according to an embodiment of the disclosure.

DETAILED DESCRIPTION

The following detailed description is exemplary in nature and is notintended to limit the invention or the application and uses of theinvention. Furthermore, there is no intention to be bound by anyexpressed or implied theory presented in the preceding technical field,background, summary, brief description of the drawings or the followingdetailed description.

Turning to the drawings, and as described in greater detail herein,embodiments of the disclosure provide systems and methods to opticallyimage an input object such as a fingerprint. In particular, a method andsystem is described wherein display pixels of a display are illuminatedaccording to various patterns. Detector pixels of the display are thenselectively used to image all or part of an input object, e.g., all orpart of the fingerprint. The images may then be further processed, suchas may be done for use in a biometric matching process and/or toreproduce a complete image. Additional advantages and features will beapparent from the disclosure which follows. By way of example, althoughthe description is generally directed to optically imaging fingerprints,it will be understood that the system and method describe a way to imageother input objects.

FIG. 1 is a block diagram of an example of an electronic system 100 thatincludes a display device 102 and a processing system 104, according toan embodiment of the disclosure. In accordance with the disclosure, thedisplay device 102 is also used as a sensor for imaging.

By way of example, basic functional components of the electronic device100 utilized during capturing, storing, and validating a biometric matchattempt are illustrated. The processing system 104 includes aprocessor(s) 106, a memory 108, a template storage 110, an operatingsystem (OS) 112, and a power source(s) 114. Each of the processor(s)106, the memory 108, the template storage 110, and the operating system112 are interconnected physically, communicatively, and/or operativelyfor inter-component communications. The power source 114 isinterconnected to the various system components to provide electricalpower as necessary.

As illustrated, processor(s) 106 are configured to implementfunctionality and/or process instructions for execution withinelectronic device 100 and the processing system 104. For example,processor 106 executes instructions stored in memory 108 or instructionsstored on template storage 110 to identify a biometric object ordetermine whether a biometric authentication attempt is successful orunsuccessful. Memory 108, which may be a non-transitory,computer-readable storage medium, is configured to store informationwithin electronic device 100 during operation. In some embodiments,memory 108 includes a temporary memory, an area for information not tobe maintained when the electronic device 100 is turned off. Examples ofsuch temporary memory include volatile memories such as random accessmemories (RAM), dynamic random access memories (DRAM), and static randomaccess memories (SRAM). Memory 108 also maintains program instructionsfor execution by the processor 106.

Template storage 110 comprises one or more non-transitorycomputer-readable storage media. In the context of a fingerprint sensor,the template storage 110 is generally configured to store enrollmentviews for fingerprint images for a user's fingerprint or otherenrollment information. More generally, the template storage 110 may beused to store information about an input object. The template storage110 may further be configured for long-term storage of information. Insome examples, the template storage 110 includes non-volatile storageelements. Non-limiting examples of non-volatile storage elements includemagnetic hard discs, solid-state drives (SSD), optical discs, floppydiscs, flash memories, or forms of electrically programmable memories(EPROM) or electrically erasable and programmable (EEPROM) memories,among others.

The processing system 104 also hosts an operating system (OS) 112. Theoperating system 112 controls operations of the components of theprocessing system 104. For example, the operating system 112 facilitatesthe interaction of the processor(s) 106, memory 108 and template storage110.

According to various embodiments, the processor(s) 106 implementhardware and/or software to obtain data describing an image of an inputobject. The processor(s) 106 may also align two images and compare thealigned images to one another to determine whether there is a match. Theprocessor(s) 106 may also operate to reconstruct a larger image from aseries of smaller partial images or sub-images, such as fingerprintimages when multiple partial fingerprint images are collected during abiometric process, such as an enrollment or matching process forverification or identification.

The processing system 104 includes one or more power sources 114 toprovide power to the electronic device 100. Non-limiting examples ofpower source 114 include single-use power sources, rechargeable powersources, and/or power sources developed from nickel-cadmium,lithium-ion, or other suitable material as well power cords and/oradapters which are in turn connected to electrical power.

Display 102 can be implemented as a physical part of the electronicsystem 100, or can be physically separate from the electronic system100. As appropriate, the display 102 may communicate with parts of theelectronic system 100 using any one or more of the following: buses,networks, and other wired or wireless interconnections. In someembodiments, display 102 is implemented as a fingerprint sensor tocapture a fingerprint image of a user. More generally, the display 102is implemented to image an object. In accordance with the disclosure,the display 102 uses optical sensing for the purpose of object imagingincluding imaging biometrics such as fingerprints.

Some non-limiting examples of electronic systems 100 include personalcomputers of all sizes and shapes, such as desktop computers, laptopcomputers, netbook computers, tablets, web browsers, e-book readers, andpersonal digital assistants (PDAs). Additional example electronicsystems 100 include data output devices (including display screens andprinters). Other examples include remote terminals, kiosks, video gamemachines (e.g., video game consoles, portable gaming devices, and thelike), communication devices (including cellular phones, such as smartphones), and media devices (including recorders, editors, and playerssuch as televisions, set-top boxes, music players, digital photo frames,and digital cameras).

FIG. 2 illustrates an example of a display 200 according to the presentdisclosure. The display 200 includes display pixels, e.g., displaypixels 202 and 203, detector pixels, e.g., detector pixels 204 and 205,a substrate 206, and a cover layer 208. Also shown is an input object210, which according to the disclosure, and as described in detailbelow, is imaged by the display 200. As described above, the display 200may be a separate device or may be incorporated as part of variousdevices including mobile phones, media devices, and any other electronicdevice 100.

The display pixels 202 and 203 are of any suitable type. By way ofexample, suitable display pixels include light emitting diodes (LEDs)and organic light emitting diodes (OLEDs). Other sources of illuminationmay be used including, without limitation, backlighting in a liquidcrystal display (LCD). Although only two display pixels are shown inFIG. 2, an array of display pixels are used which may include any numberand arrangement of display pixels. The display pixels 202 may transmitlight of the same wavelength or may transmit light of differingwavelengths (e.g., different colors). Moreover, wavelengths other thanvisible light may be used.

In general, the detector pixels 204 and 205 detect light transmittedfrom display pixels 202. Examples of types of detector pixels 204 areCMOS sensors, phototransistors and photodiodes. Thin film transistorsensors may also be used according to the disclosure.

Although the display pixels 202, 203 and detector pixels 204, 205 aredepicted as distinct elements, it will be understood that the same typeof element may be used to both transmit light and detect transmittedlight. For example, the display pixels themselves may be reverse-biasedto function as detector pixels, using LED, OLED, or another suitabledisplay technology. The display pixels can be individually set toreverse biased detector pixels, or set in a row or column fashion.Further, all of the display pixels may be addressable in a reversebiased state, or some smaller subset may be addressable in a reversebias state in order to minimize the amount of additional routingcircuitry that is included, in which case the display might include aspecial area of fingerprint sensing corresponding to those pixels thatcan be set to a reverse biased detector state. In addition, although thedetector pixels are shown on the same substrate 208 as the displaypixels, the detector pixels can be otherwise arranged within the device,such as for example, on a different plane from the display pixels.

The cover layer 208 may include a cover lens, also referred to as acover glass, which protects the inner components of the display, such asthe display pixels 202, 203 and the detector pixels 204, 205. The coverlens may be made of any material such as chemically strengthened glass,crystalline materials (e.g., synthetic sapphire), transparent polymericmaterials, and the like. The cover layer 208 may also include one ormore additional layers associated with display and/or touch screenfunctionality. The cover layer 208 may be transparent thereby allowinglight from the display pixels 204, 205 to be transmitted and observedoutside of the display 200. A top surface of the cover layer 208 forms asensing surface 212 which provides a contact area for the input object210.

The input object 210 is an object to be imaged such as a fingerprint.Generally, the object 210 will have various characteristics. By way ofexample, the object 210 has ridges 214 and valleys 216. Due to theirprotruding nature, the ridges 214 contact the sensing surface 212 of thecover 208. In contrast, the valleys 216 do not contact the sensingsurface 212 and instead form a gap 218 between the input object 210 andthe sensing surface 212. The object 210 may have other characteristicssuch as stain or ink 220 that do not create significant structuraldifferences in portions of the input object 210, but which affect itsoptical properties.

The display pixels 202 transmit beams of light within the cover layer208 and the transmitted light becomes incident on the sensing surface212 of the cover layer 208 at various angles. At certain angles, some ofthe transmitted light is reflected and some of the transmitted light isrefracted. However, light beams transmitted from the display pixel 204which arrive at the sensing surface 212 at an angle exceeding a criticalangle θ_(c) undergo total internal reflection, i.e., all light from thetransmitted beam is reflected at the sensing surface 212.

As will be appreciated, since the medium above the sensing surface 212will vary, the critical angle at various points along the sensingsurface 212 will likewise vary. For example, the ridges 214 of the inputobject 210 and gaps 218 formed within the valleys 216 of the object willhave different indices of refraction. As a result, different criticalangles will exist at the boundaries between the sensing surface 212 andridges 214 as compared to the boundaries formed by the air gaps 218 andthe sensing surface 212. These differences are illustratively shown inFIG. 2. Line 220 represents a beam of light transmitted from the displaypixel 202 at the critical angle (θ_(cv)) for a gap 218 and sensingsurface 212 boundary, and line 222 represents the correspondingreflected beam. Line 224 represents a beam of light transmitted at thecritical angle (θ_(cr)) for a ridge 214 and sensing surface 212boundary, and line 226 represents a corresponding reflected beam.Relative to display pixel 202, region 228 depicts an area on thesubstrate 206 that is bounded by reflected light resulting from lightbeams transmitted at the critical angles θ_(cv) and θ_(cr), or in otherwords is bounded by reflected beams 222 and 226.

In accordance with one aspect of the disclosure, detector pixels fallingwithin region 228, e.g., detector pixels 204, are used to detectreflected light to image part of input object 210 when display pixel 202is illuminated. With respect to the detection of ridges and valleys,region 228 is an area of relatively high contrast. The relative highcontrast occurs because light reflected from the sensing surface incontact with valleys (e.g., air) undergoes total internal reflectionwhereas light reflected from the sensing surface 212 in contact with theinput object 210 (e.g., skin) does not. Thus, light beams transmittedfrom display pixel 202 which have an angle of incidence at the sensingsurface falling between θ_(cv) and θ_(cr) are reflected and reachdetector pixels 204 falling within region 228.

In accordance with another aspect of the disclosure, detector pixels 205falling within region 230 (relative to display pixel 202) may also beused to image the input object 210. In particular, transmitted beamsfrom detector pixel 202 which become incident on the sensing surface 212with angles smaller than both critical angle of ridge (θ_(cr)) andcritical angle of valley (θ_(cv)) result in reflected beams fallingwithin region 230. Due to scattering, the contrast of reflected beamsfalling within region 230 from ridges and valleys may be less than thecontrast of reflected beams falling within high contrast region 228.However, depending on the sensitivity of the detector pixels 204 andresolution requirements, region 230 may still be suitable for sensingridges and valleys on the input object 210. Moreover, region 230 willgenerally be suitable for detecting non-structural optical variations inthe input object 210 such as stains or ink 220.

It will be appreciated that the reflected light beams detected in region228 provide a magnified view of a partial image of the input object 210due to the angles of reflection. The amount of magnification dependsupon the distance between the display pixels 202 and the sensing surface212 as well as the distance between the detector pixels 204 and thesensing surface 212. For example, if the display pixels 202 and thedetector pixels 204 are coplanar, then the distance between the displaypixels 202 and the sensing surface 212 may be equivalent to the distancebetween the detector pixels 204 and the sensing surface 212. In such acase, the partial image of the input object 210 may undergo a two-timesmagnification (2×) based on a single internal reflection from thesensing surface 212 reaching the detector pixels in region 228.

The critical angles θ_(cr) and θ_(cv) resulting from ridges 214 and gaps218 at the sensing surface 212 are dependent on the properties of mediumin contact with the boundary formed at the sensing surface 212, whichmay be affected by a condition of the input object. For example, a dryfinger in contact with the sensing surface may result in a skin to airvariation across the sensing surface corresponding to fingerprint ridgesand valleys, respectively. However, a wet finger in contact with thesensing surface may result in a skin to water or other liquid variationacross the sensing surface. Thus, the critical angles of a wet fingermay be different from the critical angles formed by the same finger in adry condition. Thus, in accordance with the disclosure, the intensity oflight received at the detector pixels can be used to determine therelative critical angles, determine whether the object is wet or dryand/or adjust the detector pixels used for capturing the image of theobject. If a wet object is detected, a user may also be notified so thatthe object can be dried before imaging.

FIG. 3 illustrates a plan view of an example of a display according tothe present disclosure wherein various display pixels (circles) anddetector pixels (squares) are located on the same plane or parallelplanes, and wherein the sensing surface lies in a plane that is parallelto the detector pixel plane and the display pixel plane. In the example,display pixel 302 is illuminated for the purpose of imaging a portion ofthe input object 210 (FIG. 2). Concentric circles 304 and 306 illustrateboundaries of a high contrast region 308, which as described above willdepend on the dimensions of the display as well as the critical anglesθ_(cr) and θ_(cv).

In certain embodiments, when display pixel 302 is illuminated, detectorpixels falling within the high contrast region 308, such as detectorpixels 310 and 312 are used to detect reflected light from the displaypixel 302 to image a portion of the input object. In other embodiments,or in combination with the collection of data from region 308, detectorpixels, such as detector pixels 314 falling within region 318 may beused.

Also shown in FIG. 3 is a second display pixel 320. Concentric circles322 and 324 illustrate boundaries of a second high contrast region 326,which corresponds to display pixel 320. Detector pixels within region326, such as detector pixels 328 and 330, are used to collect datacorresponding to the object to be imaged. In other embodiments, or incombination with the collection of data from region 326, detectorpixels, such as detector pixel 332 falling within region 336 may beused.

In the example of FIG. 3, high contrast region 308 and high contrastregion 326 are non-overlapping. It will be understood, however, thatregions 308 and 336 may overlap. In the case of overlapping highcontrast regions, display pixels 302 and 320 may be illuminated atdifferent times, as discussed in connection with FIGS. 4A-4B below.Alternatively, provision is made to distinguish the light transmittedfrom display pixel 302 as compared to the light transmitted from displaypixel 320 in which case display pixels 302 and 320 may be simultaneouslyilluminated while data is collected within their respective highcontrast regions. When display pixels 302 and 320 are simultaneouslyilluminated as part of object imaging, FIG. 3 provides an example ofobject imaging using a spatial pattern.

It will be understood that FIG. 3 illustrates only the illumination oftwo display pixels and each display pixel includes correspondingdetection regions with which data is collected for partial images of theinput object. In operation, the system and method contemplate theillumination of as many display pixels as necessary to capture enoughpartial images to make up a larger image, or complete image of theobject. It will also be understood that various display pixels may beindependently used for displaying visual information simultaneouslywhile selected pixels are illuminated for object imaging.

FIGS. 4A-4B show a series of plan views of the display according to thepresent disclosure, which illustrates an example of object imaging usinga temporal pattern. In FIG. 4A, display pixel 402 is illuminated.Concentric circles 404 and 406 identify the boundaries of high contrastarea 408 corresponding to display pixel 402. Thus, according to thedisclosure, detector pixels within the high contrast area 408, such asdetector pixels 410 and 412, are used to collect data corresponding toridges and valleys, or other surface features, from the object 212 to beimaged. Alternatively, or in combination with the foregoing, detectorpixels within region 412, which is radially inward from boundary 404,may be used.

FIG. 4B represents the same set of display pixels and detectors pixelsas FIG. 4A, but at a different time. Display pixel 414 is illuminated.As will be noted, the concentric circles 416 and 418 identifying theboundaries of corresponding high contrast region 420 have moved relativeto the high contrast region 408 of FIG. 4A. Thus, the subset of detectorpixels falling in the high contrast area have changed, although somepixels may fall with both high contrast areas 408 and 420 such asdetector pixel 412.

In the example of FIGS. 4A and 4B, the various high contrast regions 408and 420 overlap. However, illumination of the display pixels 402 and 414are temporally spaced. For example, display pixel 402 is illuminated.After the data is collected from within region 408, display pixel 402 isturned off. Display pixel 414 is then illuminated and data is collectedfrom within region 420. After data is collected from within region 420,display pixel 414 is turned off. This process continues using as manydisplay pixels as necessary to capture enough partial images to form alarger or complete image of the input object as desired. As previouslydescribed, the disclosure also contemplates the simultaneousillumination of multiple display pixels having overlapping high contrastareas provided that the reflected light received from the differentillumination pixels can be resolved.

FIG. 5 illustrates a plan view of a partial image of an objectsuperimposed onto a high contrast region 504, which is imaged duringillumination of display pixel 506. Concentric circles 508 and 510 showthe boundaries of the high contrast region 504. Portions 512 correspondto ridges of the input object. Other areas within the high contrastregion 504 correspond to valleys 518 of the input object. As previouslydescribed, due to the angles of reflection undergone by lighttransmitted by display pixel 506, the ridges and the valleys detected inthe high contrast region 504 are magnified as compared the actual ridgesand valleys on the object. The amount of magnification depends on thegeometry of the display including the distance between the displaypixels, detector pixels, and the sensing region. Moreover, detectorpixels further away from the display pixel 506, e.g., detector pixel514, will receive lower intensity reflected light as compared todetector pixels closer to the display pixel, e.g., detector pixel 516because the intensity of light decreases in relation to the distance ittravels.

In some applications, image data from various partial images obtainedduring patterned illumination of the individual display pixels iscombined into composite image data of the input object. The partialimage data may be aligned based on known spatial relationships betweenthe illumination pixels in the pattern. By way of example, the partialimage data may be combined by stitching together the partial images intoa larger image, or by generating a map that relates the image data fromthe various partial images according to their relative alignments.Demagnification of the images may be useful prior to such piecingtogether or mapping. In addition, it may be useful to apply a weightingfunction to the image data to account for the different intensities oflight received at detector pixels having different distances from thedisplay pixels. In some applications, if pixels inside of region 508 areused, the resulting data from the various partial images may bedeconvolved to reconstruct the larger image. Alternatively, the datainside of this region may convey sufficient information for someapplications, so that no deconvolution is needed.

FIG. 6 illustrates a way to provide feedback during imaging of an objectusing a display according to the present disclosure. Such feedback maybe used, for example, to provide feedback to a user during acquisitionof a fingerprint image in an enrollment and/or authentication process.

As shown, the device 600 includes a display area 604. An object 606,such as a finger, is placed over and in contact with the display area604. Display pixels underneath the object 606 are illuminated accordingto a pattern to image part or all of the object 606 in accordance withthe previous description. During or after imaging of the object 606,display pixels at or about the perimeter of the object 606 areilluminated to provide a visually perceptible border 608. The displayedborder 608 may change in appearance to signify status. For example,while the object 606 is being imaged and/or during an authenticationperiod, the border could be a first color (e.g., yellow). Once theimaging and authentication is completed, the color could change to asecond color (e.g., green) if the authentication is successful or athird color (e.g., red) if the authentication is unsuccessful. It willbe appreciated that changes in color provides one example of how theborder 608 may be altered to signal status to the user. Other changes inthe appearance of the border, such as a change from dashed line to asolid line, or an overall change in the shape of the border could beemployed as well.

FIG. 7 illustrates a method 700 of obtaining, processing and performingmatching of an image of an input object, such as a fingerprint. By wayof example, matching may be used for biometric authentication orbiometric identification. It will be appreciated that the steps andsequence of steps are by way of example only. Steps may be eliminated orthe sequence modified without departing from the system and method ofthe present disclosure.

In step 702 the presence of an input object in contact with the sensingsurface of the display is detected. Such detection may, for example,occur as the result of detection of changes of intensity in lightreceived at detector pixels in the display. Alternatively, presence ofthe input object may be detected via capacitive sensing or otherconventional techniques using a touch screen for example.

In step 704, the system and method may determine moisture content of theinput object to be imaged. The moisture content can be determined, forexample, by illuminating display pixels to determine the inner boundaryof the high contrast area. By comparing the determined inner boundary ofthe high contrast to an expected boundary for a dry object, the relativemoisture content can be estimated. The moisture content can be used forvarious purposes. For example, the detected moisture content can be usedas a metric of expected image quality. The detected moisture content mayalso be used to establish the boundaries of high contrast and,therefore, used to establish which detector pixels will be used tocollect data when a given display pixel is illuminated as part of theimaging process. The detected moisture content may also be used notifythe user that a suitable image cannot be obtained. The user may then drythe object (e.g. finger) and initiate another imaging attempt.

In step 706, one or more display pixels are illuminated to image theinput object. The display pixels illuminated and sequence ofillumination depend on the pattern used. If a spatial pattern is used,multiple spatially separated display pixels are simultaneouslyilluminated. If a temporal pattern is used, display pixels areilluminated at different times. As previously described, the patternused for imaging may include a combination of temporal and spatialpatterns. For example, a first set of display pixels may be illuminatedfirst where the corresponding high contrast areas are non-overlapping.This may then be followed by a second set of distinct display pixelsbeing illuminated which likewise provide non-intersecting high contrastregions and so on. The display pixels illuminated and sequence ofillumination may also be guided by a touch position detected bycapacitive sensor or touch screen, for example.

It is further contemplated that multiple display pixels may beilluminated even though they provide overlapping high contrast areas. Insuch an arrangement, the display pixels transmit light of differentwavelengths (e.g., colors), which can be separately detected to resolvedifferent partial images of the object. Alternatively, techniques suchas code division multiplexing may be used to transmit the light. In suchan arrangement, the collected data may be de-convolved to resolve thedifferent subparts of the fingerprint. Other methods to distinguishbetween light transmitted from different display pixels may be usedprovided that light transmitted from different display pixels can bedetected and distinguished.

In step 708, the method obtains image data from appropriate detectorpixels. The appropriate detector pixels will, for example, be thedetector pixels in the corresponding high contrast region(s) for thedisplay pixel(s) illuminated. However, as previously described, a regioninside of the high contrast region may be used.

In step 710, a determination is made as to whether the pattern iscomplete. The pattern is complete when data for all of the partialimages that will make up the entirety of a desired image of the objectis collected. If the pattern is not complete, the process returns tostep 706. In step 706, the next sequence in the process is initiated,for example, the next display pixel or set of display pixels isilluminated.

In step 712, the collected data for the various partial images undergoprocessing. By way of example, the processing may includedemagnification of the image data and/or the application of weightingfactors to the image data to account for the different intensities oflight detected at detector pixels further away from the display pixels.The processing may further include combining the data for the variouspartial images into a complete image or creating a template that relatesthe partial images to one another even though they are kept separate.The image data from the various partial images may be combined accordingto the known geometric relationships between the pixels in the pattern.The image data may also be combined based on other parameters, such asthe thickness of the cover layer, which provides additional informationabout the light beam paths from the illumination and detector pixels tothe sensing surface to resolve physical transformations between thepartial images. The thickness of the cover layer may be pre-defined ormay be computed at image capture time based on the location of the innerboundary of the high contrast region. For example, the location of theinner boundary may be closer or further away from the illuminateddisplay pixel for thinner or thicker cover layers, respectively.

In step 714, the image data may be compared to previously stored imagesof the object. For example, an image of a fingerprint taken during anauthentication attempt may be compared to previously stored enrollmentviews of the fingerprint. If a match is detected, the user isauthenticated. If a match is not detected, authentication may be denied.As another example, an image of a fingerprint taken during a controlinput may be compared to previously stored enrollment views of thefingerprint to identify which finger provided the input. If a match isdetected to a specific finger, a finger specific display response orother device operation may then be initiated based on the identifiedfinger.

As described in connection with FIG. 6, the user may be provided withfeedback during the process described in connection with FIG. 7. Forexample, a colored border may be provided around the user's fingerduring imaging and/or while the authentication process is underway. Oncethose processes are complete, the color of the border may change tosignify completion of imaging and the results of the authentication. Forexample, a green border signifies authentication is successful whereas ared border signifies that the authentication failed.

After image processing, the collected data for the object may be storedfor later use.

The use of the terms “a” and “an” and “the” and “at least one” andsimilar referents in the context of describing the invention (especiallyin the context of the following claims) are to be construed to coverboth the singular and the plural, unless otherwise indicated herein orclearly contradicted by context. The use of the term “at least one”followed by a list of one or more items (for example, “at least one of Aand B”) is to be construed to mean one item selected from the listeditems (A or B) or any combination of two or more of the listed items (Aand B), unless otherwise indicated herein or clearly contradicted bycontext. The terms “comprising,” “having,” “including,” and “containing”are to be construed as open-ended terms (i.e., meaning “including, butnot limited to,”) unless otherwise noted. Recitation of ranges of valuesherein are merely intended to serve as a shorthand method of referringindividually to each separate value falling within the range, unlessotherwise indicated herein, and each separate value is incorporated intothe specification as if it were individually recited herein. All methodsdescribed herein can be performed in any suitable order unless otherwiseindicated herein or otherwise clearly contradicted by context. The useof any and all examples, or exemplary language (e.g., “such as”)provided herein, is intended merely to better illuminate the inventionand does not pose a limitation on the scope of the invention unlessotherwise claimed. No language in the specification should be construedas indicating any non-claimed element as essential to the practice ofthe invention.

Preferred embodiments of this invention are described herein, includingthe best mode known to the inventors for carrying out the invention.Variations of those preferred embodiments may become apparent to thoseof ordinary skill in the art upon reading the foregoing description. Forexample, although a system and method are described for imaging an inputobject using a display, the system and method can be implemented as animager that is separate from the display. Moreover, although thedescription is generally directed to imaging a fingerprint, the systemand method may be used to image any object. The inventors expect skilledartisans to employ such variations as appropriate, and the inventorsintend for the invention to be practiced otherwise than as specificallydescribed herein. Accordingly, this invention includes all modificationsand equivalents of the subject matter recited in the claims appendedhereto as permitted by applicable law. Moreover, any combination of theabove-described elements in all possible variations thereof isencompassed by the invention unless otherwise indicated herein orotherwise clearly contradicted by context.

The invention claimed is:
 1. A display for imaging an input object,comprising: a sensing surface; a plurality of display pixels; aplurality of detector pixels; and a processing system configured to:determine a location of the input object relative to the sensingsurface; illuminate one or more display pixels of the plurality ofdisplay pixels according to a pattern based, at least in part, on thelocation of the input object; acquire image data of the input objectfrom one or more detector pixels of the plurality of detector pixels,wherein the image data corresponds to light from the one or more displaypixels that is reflected at the sensing surface.
 2. The display of claim1, further comprising: a touch screen to detect the location of theinput object.
 3. The display of claim 2, wherein the touch screenincludes a capacitive sensor.
 4. The display of claim 1, furthercomprising: an optical sensor to detect the location of the inputobject.
 5. The display of claim 4, wherein the processing system isconfigured to determine the location of the input object based, at leastin part, on a detected change of intensity at the plurality of detectorpixels.
 6. The display of claim 1, wherein the processing system isconfigured to select the one or more detector pixels from a regionsurrounding the one or more display pixels, the region being inside ofan outer boundary that is defined by a first critical angle resultingfrom an interface between a surface of the input object and the sensingsurface.
 7. The display of claim 1, wherein the processing system isconfigured to select the one or more detector pixels from a regionsurrounding the one or more display pixels, the region being outside ofan inner boundary that is defined by a second critical angle resultingfrom an interface between air and the sensing surface.
 8. The display ofclaim 1, wherein the pattern is time varying such that illumination of afirst set of the one or more display pixels is temporally spaced fromillumination of a second set of the one or more display pixels.
 9. Thedisplay of claim 1, wherein the pattern is spatial and whereinilluminate one or more display pixels further comprises: illuminate afirst set of one or more display pixels; and simultaneously illuminate asecond set of one or more display pixels, wherein the first set of theone or more display pixels is physically spaced apart from the secondset of the one or more display pixels.
 10. The display of claim 1,wherein the processing system is further configured to: illuminate aportion of the display, the display at least partially surrounds theinput object such that the portion is visually perceptible.
 11. Thedisplay of claim 1, wherein the plurality of detector pixels aredistinct from the plurality of display pixels.
 12. The display of claim1, wherein the plurality of detector pixels comprises reverse-biaseddisplay pixels.
 13. An electronic device for imaging an input object,comprising: a processing system configured to be communicatively coupledto a display, the processing system further configured to: determine alocation of the input object relative to the display; illuminate one ormore display pixels of the display based, at least in part, on a patternassociated with the location of the input object; acquire image data ofthe input object in contact with a sensing surface of the display fromone or more detector pixels of the display, wherein the image datacorresponds to light from the one or more display pixels that isreflected at the sensing surface.
 14. The electronic device of claim 13,wherein the processing system is configured to determine the location ofthe input object based, at least in part, on a detected location from atouch screen.
 15. The electronic device of claim 14, wherein the touchscreen comprises a capacitive sensor.
 16. The electronic device of claim13, wherein the processing system is configured to determine thelocation of the input object based on a detected change of intensity ata set of detector pixels.
 17. The electronic device of claim 13, whereinthe processing system is configured to select the one or more detectorpixels from a high contrast annular region surrounding the one or moredisplay pixels, the high contrast annular region being between an outerboundary and an inner boundary, the outer boundary defined by a firstcritical angle resulting from an interface between a surface of theinput object and the sensing surface and the inner boundary beingdefined by a second critical angle resulting from an interface betweenair and the sensing surface.
 18. A method for imaging an input objectcomprising: determining a location of the input object relative to asensing surface of a display; illuminating one or more display pixels ofthe display according to a pattern based, at least in part, on thelocation of the input object; acquiring image data of the input objectin contact with the sensing surface from one or more detector pixels ofthe display, wherein the image data corresponds to light from the one ormore display pixels that is reflected at the sensing surface.
 19. Themethod of claim 18, wherein the determining the location of the inputobject is based, at least in part, on a detected location from a touchscreen.
 20. The method of claim 18, wherein the determining the locationof the input object is based on a detected change of intensity at a setof detector pixels.